This invention relates generally to optical devices which project images on planar surface, and more particularly to projecting an arbitrary number of images to form a single mosaic displayed image.
A photo-mosaic is a 2D array of images registered together to form one large preceived image. The images are either taken from the same viewpoint in different directions or taken from different viewpoints of a planar scene, see Szeliski et al. xe2x80x9cCreating full view panoramic image mosaics and texture-mapped models,xe2x80x9d Computer Graphics (SIGGRAPH""97), pp. 251-258, 1997, Chen et al. xe2x80x9cQuicktime VRxe2x80x94an image-based approach to virtual environment navigation,xe2x80x9d Comp. Graph. Proc., Annual Conf. Series (Siggraph ""95), pp.29-38, 1995. and U.S. Pat. No. 5,986,668 xe2x80x9cDeghosting method and apparatus for construction of image mosaicsxe2x80x9d issued to Szeliski, et al. on Nov. 16, 1999.
Large projectors arrays are popular because they offer a practical solution to the problem of generating a high-resolution and bright image. Older systems, such as video walls, typically use an array of rear-projectors with abutting image edges separated by a small but always visible gap or overlap. Newer systems use overlapping projectors with facility for precise manual electro-mechanical adjustment for image registration and blending. The setting up of these displays is still quite tedious, requiring precise projector overlap, and often a near orthogonal projection to the display surface, i.e., the display surface has to be perpendicular to the optical axis of each projector. This arguably is the most prominent drawback of large format display design.
A projector is similar to a camera in that the image projection process can be expressed using the well-known pinhole camera model. Thus far, however, projectors have received little attention in the field of computer vision.
Raskar et al. in xe2x80x9cMulti-projector displays using camera-based registration,xe2x80x9d IEEE Visualization, pp. 161-68, 1999 described various approaches to building a multi-projector display system. They provided a general solution to the seamless display problem using stereo cameras to determine the display surface and individual projector""s intrinsic and extrinsic parameters all in a common coordinate frame. The result was an exhaustive description of the entire display environment. Although that approach allowed for a general solution, the computational effort and resources needed to implement that approach introduced their own level of complexity.
Chen et al. in xe2x80x9cAutomatic alignment of high-resolution multi-projector displays using an UN-calibrated camera,xe2x80x9d IEEE Visualization 2000, 2000, provided a mechanism to reduce the problem of mechanical alignment using a camera with controllable zoom and focus, mounted on a pan-tilt unit. The data collection and computation took over thirty minutes. Surety in xe2x80x9cScalable Self-Calibration Display Technology for Seamless large-scale Displays,xe2x80x9d pH Thesis, Massachusetts Institute of Technology, 1999 presented a solution that also used a camera to establish the relative geometry of multiple projectors. The camera was calibrated by imaging a piece of paper printed with a regularly spaced grid placed in front of the display surface. Subsequently, projector imagery was registered to the grid.
It is desired to significantly reduce the support and cost for projector arrays. It is also desired to provide a flexible image registration and rendering technique that can adapt to a given projector array configuration. Furthermore, it is desired to fully automate and reduce the time required for setting up and registering multiple projectors.
The present invention provides a multi-projector display system having an arbitrary number of projectors. An automated method is provided for registering the projector array to produce a seamless single rectangular displayed image. A single camera is used to determine the pose of the projector from the relative location and orientation of each projected image. The poses of the projectors are used to register the projectors with respect to the planar display surface to achieve registration and intensity blending of the images. During operation, source images are warped so that they appear correctly as a single image when projected onto the planar display surface. The warped images are then intensity corrected to account for overlap. The corrected images can then be projected. As an advantage, the display surface can be oblique to the optical axes of the projectors.
More specifically, the invention provides a method and system for forming a mosaic image on a display surface with a multiple projectors. For each projector in turn, a registration image is projected onto the display surface so that a union of the projected registration images forms a polygon.
With a camera, for each registration image in turn, a corresponding input image is acquired. A display area on the display surface enclosed by the polygon is then identified, and a single projective matrix between the display area and each input image is determined for each projector.
A source image for each projector is warped according to the corresponding single projective matrix of the projector. The pixels of the warped source image are weighted according to the projective matrix, and then the warped and weighted source images are concurrently projected directly onto the display surface to form the mosaic image.